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Introduction 

Teachers are effected directly by state standards and assessment policies, as they are 
presented with new expectations for what their students will learn and new mechanisms for 
determining if students are meeting these expectations. However, much of the impact of these 
policies is mediated by the outlook of, and decisions made by, district administrators about how 
to attempt to alter teacher practice in ways consistent with reform goals (Spillane, 1996). 
Literature on the implementation of reforms has repeatedly shown that local agents 
interpretations and responses to policy changes have a profound effect on policy impact (Berman 
& McLaughlin, 1977; Fairman & Firestone, 2001; Spillane & Thompson, 1997). 

Supporters of using state assessment to “drive” instructional change suggest that high- 
quality tests will promote higher quality instruction (Resnick & Resnick, 1992). New Jersey’s 
Elementary School Proficiency Assessment (ESPA) is a test that relies not just on multiple 
choice questions, but also on item that expect students to be able to demonstrate more conceptual 
understanding and explain their responses; this is consistent with some ideas of a “good” test. 

In order to understand the impact of New Jersey’s standards and assessment policies, we 
explore how district leaders responded to the state’s standards and the fourth grade ESPA, 
specifically in the area of mathematics. Cohen and Barnes, among others, have argued that 
policy is itself an instrument for teaching and learning (Cohen & Barnes, 1993). Thus, district 
administrators who are adapting district efforts in response to New Jersey’s standards and the 
ESPA must determine what their “pedagogy” will be for promoting student learning linked with 
new state expectations by altering the actions and knowledge of teachers. Building on earlier 
work exploring district responses to testing (Fairman & Firestone, 2001), this paper explores the 
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relationship between district pedagogy around testing and the will and capacity in particular 
districts, including the physical capital (resources), human capital and social capital (Spillane & 
Thompson, 1997). The use of state assessment as an important tool to drive instructional change 
relies on multiple levels of the system to respond with substantive - rather than cosmetic - 
change. Literature on district reform suggests that district capacity - especially human capital - 
may have important implications for how the district level responds to these new demands. 

Literature review and conceptual framework 

District Responses to Testing 

There are a variety of ways in which districts can respond to standards and assessments. 
Cohen and Barnes explore the “pedagogy of policy,” or how policy makers have sought 
(consciously or not) to “teach” implementers of educational reforms; they conclude that, “the 
pedagogy of policy has been didactic and inconsistent” (Cohen & Barnes, 1993, p. 226). 
Supporters of standards-based accountability argue that the implementation of aligned standards 
and assessments can promote increased student learning (Smith & O'Day, 1991). Districts 
developing pedagogy in line with such ideals need to emphasize and support teacher learning 
that is consistent with the student learning sought by the standards and assessment. However, 
districts can also adopt “quick fixes” through their pedagogy, targeting their efforts in directions 
they believe will lead to improved test scores without placing as much emphasis on the 
fundamental learning that the tests are designed to promote. 

Literature on the district role has primarily focused on reform in general (c.f Firestone, 
1989; Floden & others, 1988; Spillane, 1998b), while literature on the impact of testing has 
emphasized teachers’ interpretations and implementation efforts at the classroom level (Fairman 



Teaching the Test to the Teachers 



2 



DRAFT 



& Firestone, 2001). Districts must develop or utilize instructional guidance instruments in order 
to encourage teacher change in response to testing. In terms of general reform efforts, Spillane 
identifies four “formal channels” used by district leaders to shape classroom teaching and student 
learning - curriculum guides, materials, student assessment and professional development 
(Spillane, 2000). In terms of professional development, districts vary in the amount of 
professional development provided, the depth of content and continuity, and the level of 
“centralization” of these offerings (Little, 1989). Additional “pedagogical tools” could include 
using other resources, including altering the role of staff or the use of time, and providing test- 
specific information to students and parents. The theory underlying standards-based reform 
suggests that these policy levers should be aligned with one another, and with state standards and 
assessments (Smith & O'Day, 1991). 

The process that districts choose to use in response to testing reflects another set of 
pedagogical choices. Spillane discusses the value of teacher change and learning that builds on 
and challenges teachers’ current thinking, and links the use of teacher leaders as central agents in 
the change process to the development of such learning (Spillane, 2000). The role of teachers in 
reform efforts is reflected in the tools or policy levers used. As Floden and his colleagues 
discuss, some districts encourage teacher participation and design policy levers that encourage 
teacher involvement (i.e. in the district plaiming process or as teacher leaders), while others 
focus more on the decisions of district administrators with little teacher input (Floden & others, 
1988). 

In terms of responses to testing more specifically, Fairman and Firestone found that, in 
Maine and Maryland, the districts and teachers they studied mostly responded to state standards 
and assessments by doing more test practice activities in the format of the new assessments and 



Teaching the Test to the Teachers 



3 



DRAFT 



adding new topics to the curriculum, rather than focusing on fundamental changes in pedagogy 
(Fairman & Firestone, 2001). 

Finally, it is important to attend not only to what tools are being used in response to testing 
demands, but whether these tools constitute an overall coherent and consistent strategy, or are 
merely “piecemeal” changes (Floden & others, 1988; Porter, 1989). As well, in high capacity 
districts, Spillane and Thompson found that administrators and teacher-leaders, “frequently saw 
their task as helping their colleagues learn key reform ideas, rather than telling them what the 
reforms were about and forcing them to change” (Spillane & Thompson, 1997, p. 192). Thus, 
there was variation in the process of change (top-down versus collaborative with teachers). 

Why do districts respond to state standards and testing in different ways? 

Research on districts offers some fairly consistent findings on the reasons behind district 
responses to reform in general, as well as to testing more specifically. The concepts of district 
will and capacity are particularly useful. 

District Will 

Several authors have discussed the “will” that is necessary for local agents, including district 
administrators, to implement change consistent with reform ideas; such motivation is seen in part 
as the result of the “fit” of district policy with reform ideals (Firestone, 1989; McLaughlin, 

1987). Fairman and Firestone argue that individual will to develop new knowledge is similar to 
professional commitment, and requires some ability to understand the nature of the change 
sought by the policy (Fairman & Firestone, 2001). 

According to Fairman and Firestone, state policy interacts with districts’ and teachers’ will 

and capacity to influence instmctional change (Fairman & Firestone, 2001). They found that 
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district will depends on some mix of the nature of the state policies (such as stakes or other 
accountability tools), size of district, governance structure of the districts, and the values of 
individual administrators. The “will” of district leaders to seek changes consistent with reform 
more generally is closely tied to their own understandings of reform ideas (Spillane, 1996; 
Spillane, 1998a). Fairman and Firestone also found that, while some district leaders felt 
compelled to comply with state mandates, others felt more independent and disagreed with or did 
not understand the premise of the reforms. 

The support of district leaders for reform linked to testing is also likely influenced by other 
demands on their time and by the level and nature of community pressure to improve test scores. 
For example, if the district has already committed to implementing certain curricular or testing 
changes, then administrators may be reluctant to redirect their energy and money when a state 
policy change comes about (Spillane, 1996). Spillane also notes how, in one suburban district he 
studied, “community interest in test scores was especially influential in the suburban assistant 
superintendent’s efforts to make sense of the reforms” ( see also Fairman & Firestone, 2001; 
Spillane, 1998a, p. 50). Fear of embarrassment over public comparisons of districts test scores 
was one reason that some administrators in Maine and Maryland felt compelled to give more 
attention to math topics tested by the state or teach test taking skills (Firestone, Mayrowetz, & 
Fairman, 1997). While administrators’ beliefs about, and understanding of, reform ideas is 
important, their commitment to change can also be cmcial (Spillane & Thompson, 1997). 

District Capacity 

The idea of district “capacity” to support reform, whether in general or specifically around 

standards and assessments, has been raised by a number of researchers (Fairman & Firestone, 

2001; Spillane & Thompson, 1997). Spillane and Thompson focus on three particular aspects of 
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capacity - physical capital and resources, human capital, and social capital - recognizing that 
these “capitals” are heavily intertwined (Spillane & Thompson, 1997). Here, we also focus on 
these three aspects of capacity, but look more specifically at how they are linked to state 
standards and assessments. 

Human capital. Human capital in the context of district leadership can be defined as “the 
knowledge and skills that administrators and teacher leaders command. . .[along with] the 
commitment to reform and the disposition to leam” (Spillane & Thompson, 1997, p. 190). 
Recent research has demonstrated the importance of administrators’ beliefs about teacher 
learning in overall reform efforts. For example, Spillane highlights how the views of individual 
administrators on instruction influence their interpretations of state directives (Spillane, 1996; 
Spillane, 1998a). 

These views are particularly important when it comes to the connections between state 
testing and district leaders’ ideas about reform. Spillane found that some of the districts he 
studied in Michigan viewed the state assessment as external to their reform efforts (Spillane, 
1996). Administrators may value their own locally chosen assessments more than the state’s, 
which may give teachers conflicting messages about what skills to focus on in instruction. It is 
also important to note that districts do not necessarily have a cohesive district “vision” related to 
reform efforts; as Spillane points out, there can be within-district variation in beliefs about 
reform among key leaders at the district and school levels (Spillane, 1998b). 

Spillane and Thompson comment that, in their study, “Invariably we found that the most 
successful local reformers were knowledgeable about subject matter as well as about current 
thinking on the teaching and learning of these subjects” (Spillane & Thompson, 1997, p. 192). 



Teaching the Test to the Teachers 



6 



DRAFT 



The views that district administrators hold may be linked to convictions about teaching 
based on their own prior classroom experience and training; their individual convictions may be 
a more powerful influence on decision-making than state policy messages (Spillane, 1996). 
Spillane notes that, “Work in organizations is also shaped by the particular specializations and 
professional or occupational identities individuals bring to their work. Individuals in these 
professional communities share norms, knowledge, perspectives, commitments, and often a 
language or vocabulary” (Spillane, 1998b, p. 37). 

Social capital 

Spillane and Thompson found, in the districts they studied, that “social capital in the forms 
of professional networks and trusting - collegial relations - was instrumental to the creation of 
the human capital necessary to realize the reform ideas” (Spillane & Thompson, 1997, p. 190). 

Networks that bring ideas and support from outside a district, including links with 
universities and professional organizations, can provide an important resource for district 
administrators trying to understand reform ideas and determine how to approach district change 
(Spillane, 1998a; Spillane, 1998b). Spillane comments that, “Professional associations are likely 
to be important in most medium and large district offices where one is likely to find an array of 
professional specializations, in subject matter or assessment. . . Organizational arrangements and 
professional affiliations help situate local policymakers’ efforts to understand state policy” 
(Spillane, 1998b, p. 37). 

Spillane and Thompson argue that, “developing social capital involves changing the way 

people relate with each other in order to enable them to achieve goals that could not be possible 

in the absence of these relations” (Spillane & Thompson, 1997, p. 193). They find that links to 

formal and informal networks - and the use of those networks - was critical in high capacity 
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districts. This was particularly true in smaller districts, where internal human capital was often 
more limited (Fairman & Firestone, 2001). In this study, all of the districts had access to the 
New Jersey Statewide Systemic Initiative as one network for building social capital 

Physical Capital/Resources. Physical capital (which Spillane and Thompson discuss under the 
heading of “resources”) can include staffing, time and materials (Spillane & Thompson, 1997). 
Decisions about use of these resources are heavily influenced by a district’s human capital. As 
well, district size can have an important impact on capacity, as larger district have more 
resources to draw upon and can have district staff that are more specialized (Fairman & 

Firestone, 2001). 

Conceptual Framework and Research Questions 

As the above literature review demonstrates, district will and capacity can have an important 
impact on district responses to reform. In this study, we draw on many of the same ideas to look 
more specifically at the relationships between will and capacity and district responses to state 
standards and testing. Specifically, we ask: 

How do districts respond to state standards and testing? What tools and methods do 
they use to promote change in this context? 

How does district capacity influence districts’ responses? 

Methods and Study Context 

Data Collection and Analysis 

This study involves an examination of the responses to state standards and the ESPA in six 
districts in New Jersey. This study is part of a larger study that explores the relationship between 
state testing, professional development and instructional practices in mathematics and science. 
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The districts were selected through recommendations from leaders at three of the New Jersey 
Statewide Systemic Initiative Regional Centers. Districts were purposively sampled with the 
intent of focusing on districts that were actively engaged in math or science reform efforts, and 
represented a variety of contexts (district size, urban/rural/suburban, and district affluence). 

Table 1 shows the district characteristics in these areas. 

The study is a comparative case study (Yin, 1994) of six districts, and the primary data used 
were interviews with district administrators. In total, 19 administrators were interviewed. Of 
these, 1 1 were interviewed once, and the remaining eight were interviewed twice. Interviews 
lasted between 45 minutes and one and a half hours. Following the interviews, tapes were 
transcribed and district profiles were created. These district profiles, along with the original 
interviews, were coded using the Nud*Ist qualitative data software program, based on themes 
that emerged through the literature and in the process of conducting the study. Following Miles 
and Huberman, the coded data was then summarized and placed in data matrices based on the 
themes utilized in this paper (Miles & Huberman, 1994). 

Study Context 

This study takes place in a single state. New Jersey, where standards-based reform has taken 
a relatively moderate form. New standards were developed, and tests aligned with these 
standards have been implemented in grades 4, 8 and 12. The single state nature of the study 
offers an important advantage for a study of district responses to standards and assessment 
policies, as all six districts are operating within a somewhat similar external context. 

However, New Jersey is also a somewhat unique state, in that litigation linked with issues of 
finance and educational equity has a significant aspect in some districts. Thus, one contextual 
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factor that may reasonably be expected to influence both district “will” and capacity for reform is 
whether or not the district is an “Abbott” district. This set of 28 primarily urban, low-income 
districts have been identified as high needs through a series of state supreme court decisions. As 
a result of these decisions, the Abbott districts receive additional funds to improve educational 
quality, but are also under additional requirements from the state. During the time of this study, 
the Abbott districts were in the midst of implementing mandatory whole school reform models in 
their schools - this had substantial implications for a district’s ability to take “district” action, as 
budgeting and professional development decisions were being substantially shifted to the school 
level. For the most part, the efforts described here undertaken in the two Abbott districts 
included in this study (Cedarville and Ridley) reflect district strategies prior to the shift towards 
whole school reform. 

Analysis 

Strategies for approaching standards and testing 

Responses to standards and testing varied from district to district, both in the “tools” used 
and how those tools were used (i.e. professional development, assessment, and curriculum 
change), and the process of creating and supporting district-wide change. Table 2 shows a 
matrix reflecting the different strategies around tools and process used in the six districts studied. 

Tools Used in Change 

The six districts we studied approached change linked with state standards and the ESPA in 
a variety of ways, using tools including professional development, curriculum change, 
assessment, new resources and changed staffing patterns. Each district used a somewhat 
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different combination of the tools available, and used these tools in different ways (see Table 3). 
However, upon closer inspection, we were able to identify a continuum of strategies implicit in 
the tools selected. On one end of the continuum are strategies that rely primarily on tools 
intended to raise test scores through “test-prep” activities, such as training teachers to better 
prepare students to be effective test-takers through exercises, materials and/or assessments 
designed to make students more comfortable with the test format and context and altering the 
content of the curriculum to reflect topic areas that are “weak.” On the other end of the 
continuum are in-depth reform-oriented strategies designed to develop teachers’ content and 
pedagogical knowledge consistent with the goals of the New Jersey standards. 

Professional development 

Professional development was used in all six districts as they sought to respond to new state 
expectations. However, the intensity, follow-up and content of this professional development 
varied considerably. In Newtown, which had the most “test-prep” response, professional 
development linked to standards and testing was solely provided by outside experts and district 
staff, and focused on issues such as how to correlate standards with the curriculum. Professional 
development in this district is primarily “one-shot” workshops, and there is little in the way of 
follow-up (with the exception of teacher observations, where teachers are expected to have 
lessons that are linked to the standards). Willis Creek also focused primarily on “test-prep” 
activities, but supplemented workshops with some classroom follow-up. 

In the “combination” category, districts used professional development with a mixture of 

goals, some aimed to enhance teachers’ knowledge and understanding, and others designed to 

prepare teachers to respond to specific issues of content and test-taking strategies. These 

districts required some attendance at workshops, and also offered more in-depth opportunities for 
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interested teachers (such as courses for university credit). Ridley, the most “heavy-handed” 
district in this area, required teachers to demonstrate classroom use of workshop concepts before 
they received all compensation for workshop participation. Sunset is the one district where the 
overall response to state expectations has been the most consistent with promoting substantive 
teacher learning. Here, professional development in the district’s ongoing, intensive 
mathematics reform effort has not been altered considerably in response to state policy, but some 
supplemental workshops on standards, ESPA and testing strategies have been added. 

Assessment 

In most districts, analysis of test scores from the ESPA and supplemental standardized tests 
were used to identify areas of weakness and the district’s curriculum or “signals” to teachers 
about areas to address were altered to enhance instruction on topics seen as inadequately 
covered. In Willis Creek and Newtown, this was the full extent of the use of assessments in 
change efforts. Other districts, however, viewed assessment as a richer tool and used this 
strategy in conjunction with additional uses of assessment. 

Two districts (both low-income “Abbott” districts) were the heaviest users of assessment - 
adding both standardized tests and district assessments. In Cedarville, results on district 
assessments are tracked at the district level. According to the math supervisor: 

We feel that these [tested areas] are the main ideas at each grade level and we try to 
monitor the growth the student is making. . . What we are expecting our teachers to do is, 
depending on how the student scores on this, to then start spiraling their teaching. So, 
recovering skills so the next time they take it they do better. It is a way to monitor how 
we are covering the different strands and how our curriculum is doing. Then, all that data 
is sent back to me. . . Again, it is giving us a way of instant feedback of how things are 
going. 

Sunset has taken a more “bottom-up” approach to assessment than the other districts. While 
the district is giving the Terra Nova test in pre-ESPA years, they are also working to utilize new 
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teacher-developed classroom assessments that are a part of their overall reform strategy to 
promote student experience with ESPA-like testing questions and strategies. Teachers, who are 
being trained in developing alternative classroom assessments, are asked to present them in a 
format and style similar to those in the ESPA. All of the other districts are also encouraging the 
use of more open-ended classroom assessments, but not in the systematic or in-depth manner 
found in Sunset 

Curriculum and Materials 

In each district, curriculum have been realigned to match the state standards and testing 
guidelines; the level of revision varied, depending in part on how “reform-oriented” previous 
versions of the curriculum were. In terms of materials, all of the districts have purchased new 
materials for mathematics in recent years - in four districts, this has meant new textbooks, while 
the remaining two (which are moving to non-textbook based math programs) have bought other 
materials and tools. Newtown and Willis Creek were most focused on test-prep materials; 
Newtown specifically purchased test-preparation materials used in neighboring districts that had 
scored better on the ESPA. 

Stafflng and time changes 

Making changes in the use of staff and time during the school day can enable districts and 
schools to offer different types and intensity of instruction, and to enhance human capital to 
support change. In Newtown, there were no changes in staffing patterns or use of time in 
response to the state standards and the ESPA. Other districts, including Willis Creek, Hillview 
and Ridley, made fairly minor changes in staffing and did not alter the school day. Cedarville 
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had the most substantial staff changes, where three “teacher coordinators” for mathematics were 
added to the district staff. 

In terms of time, Sunset provided additional time throughout the year for mathematics to 
support the effort to offer more constructivist-oriented instruction. On the other end of the 
spectrum, Willis Creek’s superintendent explicitly stated that, “two weeks before the test is 
given, everybody stops functioning and just works on teaching those test skills.” While other 
districts likely emphasize testing skills in the weeks leading up to the ESPA, this was the most 
open and explicit allusion to such an effort. One shift reported in many districts, while not 
formalized through curriculum or official time allocations, was towards instructional times in 
areas where test scores were weaker (especially language arts and mathematics) and away from 
areas that are either untested or were district scores were seen as acceptable (often the case with 
science). 

Other tools 

There were a number of tools used in addition to those discussed above. Four districts 
offered additional instructional time, before or after school, or during the summer, to students 
that were identified as likely to do poorly on the ESPA. Several districts also used observations 
and administrator “walk-throughs” in classrooms to check on compliance with district policies. 

Community links 

Making real and substantive change in mathematics instruction, as envisioned in the New 
Jersey Core Curriculum Standards and the ESPA, could place districts at odds with communities 
that are more accustomed to traditional approaches to math. The two test-prep oriented districts, 
which made little substantive change, also did little to explain to parents and their communities 
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their approach to state initiatives. The remaining four districts used a variety of strategies, 
including from state-supported FANS (Families Achieving the New Standards) parent 
workshops, additional sessions for parents on mathematics reform, and one district’s cable access 
show that offered parents demonstrations on how to help their children with a “new” kind of 
math homework. 

Process of Change 

In addition to varying in terms of tools and overall strategies, districts also varied in the 
process they used to promote change (see Table 4). Some districts emphasized a “top-down” 
model, consistent with the teacher-driven pedagogy often used by policy makers and discussed 
by Cohen and Barnes (1993). Others combined district-driven change with a heavier reliance on 
teachers and teacher leaders to support and design the district response. No district in our sample 
relied primarily on a teacher-driven response to standards and testing. In at least one district, 
Ridley, administrators expressed a strong interest in building internal capacity and relying more 
on teacher leaders, but felt that all but a few teachers were uninterested in playing an active role 
in district efforts. 



As demonstrated here, the six districts in this study used a range of tools to respond to new 
state expectations around standards and the ESPA, and used those tools in a variety of ways. In 
some cases, tools were used as a mechanism primarily designed to improve scores, while in other 
cases efforts were focused more on substantive learning. 
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Links Between District Wiii and Capacity and Responses to State Standards and 
Testing 

In order to better understand the strategies used by districts to respond to the New Jersey 
Core Curriculum Standards and the ESPA, we examined administrators’ explanations of district 
will, or motivation to change, district capacity (including human capital, social capital, and 
resources/physical capital), and barriers or perceived barriers to change (see Table 5). Consistent 
with work by Spillane and Thompson, among others, we found that these different pieces were 
highly inter-related. While we separate them for analytic purposes, it should be visible 
throughout that the different aspects of will and capacity are interwoven. 

District Will 

District will, or motivation to seek change in response to state standards and the ESPA, was 
coimected with both internal and external pressures. Externally, will was largely linked to 
pressure from the community (especially as reflected through the school board) and parents to 
improve test scores. Most of the districts, independently of their process of change and tools for 
change, expressed that test scores are influenced their motivation because of the public image 
aspect, the pressure from parents who demand better test score and the embarrassment of being 
scoring lower than expectations. 

Community and parental pressure is found in all six districts studied. In New Jersey, DEG 

(district factor grouping) is a summary of the socio-economic status of a district and can range 

from A (very low socioeconomic status) to J (high socioeconomic status). DFG is frequently 

used for comparisons with neighboring districts. For example, in Newtown, an administrator 

commented that, “Opening day here, in September... the [superintendent] puts the scores up on 

the overhead and.. .she supplies all of the scores of all the neighboring “I” districts.” In a number 
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of districts (Sunset, Willis Creek, Hillview), changing district demographics that are making the 
districts less affluent and more diverse are adding an additional pressure as administrators must 
explain if and when scores go down (especially relative to neighboring districts) in conjunction 
with these changes. 

This external parental/community pressure appeared to be stronger in more affluent districts, 
where comparisons with other districts were frequent. As a Sunset administrator commented, 
“It’s a wealthy community. . . In a district like this, we don’t have the luxury of not doing well on 
standardized tests. It’s extremely important to our parents how well their children are doing in 
comparison to like districts.” However, middle and lower-income districts are not immune to 
such pressures. For example, the math coordinator in Hillview commented that, when the test 
scores were published, the “board just flipped. They really did. The board’s bottom line is test 
scores.” According to the superintendent, “they [board members] take it very personally when 
the scores come out comparing [us] with other districts. . . I had a board member actually shake 
his finger at me and say, ‘I demand better test scores.’” In the lower income districts, especially 
the Abbott districts, the external demands comes from a combination of community pressure and 
concern about greater state involvement (i.e. state takeover, which has happened to three 
districts). 

The demands from outside the district structure were often translated into internal pressure 
to improve scores, especially from superintendents who must respond to school board questions 
about test results. As one supervisor in Cedarville commented, “We are very concerned about 
bringing up our test scores. Our superintendent is very focused. He wants to raise 
achievement.” District will was clearly elevated by external pressures in these cases. However, 
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internal pressures/expectations can also influence the level of response to state standards and 
testing. 

Administrators in several districts talked about their support for the thinking and ideas that 
were the underpinnings of the state efforts. For example, they spoke of using the testing and 
standards to convince other district officials, teachers and parents of the need to move in more 
“reform-oriented” directions. However, a number of administrators in districts that used the 
spectrum of responses to the ESPA raised questions about the validity of this particular test. 

The pressure to raise test scores and the desire to move towards content and pedagogy 
consistent with the standards were somewhat divorced in all of the districts. Thus, all districts 
used at least some test-preparation or test-besting materials, even though administrators 
questioned their educational value. Information from other districts about success in raising 
scores through the use of such materials and strategies provided further support for what 
administrators saw as educationally questionable practices. 

District Capacity 

The human and social capital in a district, and the resources available for responses to 
changing state expectations, are heavily dependent on each other. For example, districts with 
substantial human and social capital are probably more likely to receive grants to support 
capacity-building because they have people with knowledge of grant availability and the time 
and understanding needed to successfully seek grants. In turn, the availability of resources, 
whether through district funding or external funding, can be critical in building capacity. 
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Human Capital 

Human capital, or the availability of individuals who can support change efforts in response 
to state standards and testing, involves people who are both able to devote time and effort and 
have the knowledge and understanding necessary to use their energies to move the district in the 
directions envisioned by state leaders and supporters of standards-based reform. Districts with 
more “sophisticated” responses to state standards and testing had more human capital. This 
human capital was at the district level but, especially in the most reform-oriented district, was 
also throughout the teaching staff. 

The two districts that responded to state standards and testing primarily with efforts to 
increase scores through content changes and test-prep activities had only one person at the 
district level to support change, and that person was spread across multiple content areas. In 
both cases, neither the superintendent nor the administrator focused on mathematics had a strong 
background in the content area. While the most reform-oriented district also only had one 
district staff person working with mathematics and other content areas, the district staff person in 
Sunset was highly experienced in the area and had an exceptionally strong and well-trained core 
of teachers interested in mathematics reform on which to draw. The three larger districts, which 
all took a multi-pronged attack approach, had district staff focused just on mathematics. 

At the level of district administrators beliefs and understandings, two aspects were 
particularly important in influencing the overall district response: their beliefs about what (if 
anything) teachers need to learn in order to improve test scores and how they believe teachers are 
best supported in this learning. In terms of what needed to be learned, district administrators 
varied considerably. In Newtown, the most test-prep oriented district, the curriculum director 
suggested that teachers had little to learn; in commenting on his frustration with the ESPA, he 
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argued that, for teachers, “It just means pressure and embarrassment and a sense of futility 
because I think in about 99% cases, they’re doing the very best they can. . . .and it’s a bit of an 
anomaly to me why we would do badly [on test scores].” The main changes he described as 
needed involved teaching strategies - more hands-on work, more use of manipulatives, and more 
“active learning” - that he did not connect with broader ideas of promoting students conceptual 
understandings. 

Districts with a mixed process of implementation and with a non-exclusively testing 
pedagogy seemed to have a more coherent discourse about what good teaching and learning is 
and a better understanding of the standards. In Cedarville, the math coordinator argued that 
teachers had much to learn; she said that, “the teachers don’t have the content knowledge. They 
need to know what they’re teaching. . .Especially when you have to start doing activity-based 
instructions. [Teachers] really have to have control [of content knowledge].” Similarly, in 
Ridley, the curriculum director and superintendent felt that teachers need good content-based 
courses, such as the college-level math education courses they offer in the district. While 
resources sometimes limited districts’ ability to follow through on administrators’ ideals, but an 
expectation that teachers’ pedagogical and content knowledge needed to be supported was a 
necessary (if not sufficient) condition for a more system-wide response to testing in these 
districts. 

In addition to variation in ideas about what teachers need to learn, district administrators 
also vary in how they think it is best to support and promote teacher learning. In most of the 
districts, ad mi nistrators saw a need to “coax” or require teachers to take more responsibility, get 
more actively involved in district professional development or curriculum efforts, or pursue 
professional learning (pressure, incentives, or accountability tools needed). This was particularly 



Teaching the Test to the Teachers 



20 



DRAFT 



true in districts where administrators had a perception of low human capital and motivation 
among teachers. In these cases, they were more likely to use “carrots and sticks” to promote 
participation in efforts linked with testing and standards. 

In two districts (Newtown and Ridley), administrators were explicit in arguing that teachers 
learn under pressure, and that this is why they were increasing methods of “control” (i.e. by 
mandating participation and compliance). District administrators in places where more system- 
wide changes were sought were more likely to see teacher learning as a long-term process 
requiring a variety of strategies (including formal professional development, teacher 
collaboration, and in-class follow-up). For example, an administrator in Ridley argued that, for 
real change “I really think the only way to do that is over a long period of time. Having a math 
specialist working with a small number of teachers on an ongoing basis. A privilege that we 
don’t usually have.” 

Social Capital 

As in Spillane and Thompson’s study, social capital - both internal and external - was 

strongly influenced by human capital (Spillane & Thompson, 1997). Overall, in these districts, 

social capital was necessary to support change efforts, but was insufficient without human and 

internal social capital in the district that was also supportive of such change and sought external 

links. The two “test-prep” districts had the fewest external connections around mathematics, and 

primarily relied on a few external professional development providers and a little assistance from 

the New Jersey SSI. The three districts that had a “mixed” response to testing and standards all 

had external partnerships that were supportive of more reform-oriented responses (including 

more substantial links with the New Jersey SSI); these partnerships provided networks between 

district staff and district teachers and external reform organizations. Ridley had the most 
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extensive external networks, drawing in partners from universities, reform organizations and 
businesses. 

While Sunset did not have the number of partners found in some of the other districts (i.e. 
Ridley), the connections between the district and a local university were very rich and deep. For 
example, a number of people in the district had done graduate level work at the university with 
the same faculty and staff who were now coming to support the district’s refomi efforts. As 
well, the curriculum director had particularly strong ties with the state and state math 
organizations. 

Social capital involves not only external relationships, but also networks and relationships 
among those within a district. Efforts such as study groups (Cedarville) and additional after- 
school time for teacher collaboration (Sunset) provided forums for teachers to share ideas and 
support. As well, using internal resources such as teachers to provide professional development 
or support to their peers also enhanced internal social capital. Overall, those districts with more 
human capital internally used this for professional development and to continue to support 
district efforts. 

Physical Capital/Resources 

The availability of resources other than people to support change efforts also varied across 

these districts. One obvious type of resource is general district funding. However, in our 

sample, the districts with more financial capacity (higher income "I" districts for instance) did 

not necessarily pursue reform more aggressively than low or middle-income districts, nor did 

their administrators and teachers have the will, attitudes, dispositions, or human capacity 

(knowledge, skills) to pursue reform aggressively on their own. If they were doing "well enough" 

on test scores, than there may not have been anything prodding them to pursue more substantive 
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changes. Only if their test scores dropped with respect to their DFG did they feel embarrassment 
from the comparison, and feel pressured by parents/school board to change "something", though 
they seem at a loss to know what to do. 

The availability of money in addition to regular funding came primarily from two sources - 
grants (usually from the govenunent or foundations) and extra funding for the two Abbott 
districts. The districts that had substantial external funding were also those that had the human 
capital that could pursue such resources. The Abbott funding was an important asset for those 
districts; the superintendent in one commented that, “the number one benefit to this school 
district in this last, I would guess 40 years, has been the Abbott Supreme Court decision.” 
However, the Abbott money can be a mixed blessing. Near the end of this study, the two Abbott 
districts were actively implementing one of the court mandates, the adoption of whole school 
reform models in each school and the shifting of district resources towards school-based 
budgeting. In the context of district reform efforts, such changes stopped or altered much of the 
district-level response to standards and the ESPA. 

While other studies have suggested that small district size can create more challenges for 
reform (Fairman & Firestone, 2001), it is interesting that the most system-wide response to 
standards and the ESPA came from one of the smallest districts. In this case, the limitations of a 
small district staff were overcome by substantial and deep capacity among the teaching staff in 
an affluent district. 

Administrator-Identified Barriers to Reform 

From the perspective of administrators, the three most commonly mentioned barriers to 

pursuing change consistent with the standards and state assessments involved teachers, the 

community and resources. Issues around staffing were mostly raised in the lower-income 
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districts of Ridley, Cedarville, and Hillview. In these districts, problems with staff quality and 
teacher turnover were raised by a number of administrators. As well, teacher beliefs and 
resistance to change were also identified as a barrier to reform. For example, an administrator in 
Ridley argued that. 

We have problems with our belief system in terms of many of the staff members not 
having a sense of efficacy about their ability to influence the development of all children 
towards high standards and the standards-based curriculum. Teachers do not look at our 
children and, this is a generalization, but it's a large enough generalization to be an issue 
in our district, teachers do not look at themselves as having the "Right Stuff. . . to 
overcome the social pathology that they attach to the kids and the kids' learning capacity. 

For those districts that were pursuing reform efforts that were non-traditional (i.e. a non- 
textbook based math program), ongoing efforts were needed to demonstrate to parents and the 
community the value of such an approach. In other districts, including Newtown and Willis 
Creek, administrators said that they were not pursuing more reform-oriented approaches to 
mathematics because of anticipated community resistance; using a strategy such as that in Sunset 
to build parental and community support did not appear to under serious consideration in these 
districts. 

Finally, even in districts where administrators had an understanding of standards and testing 
consistent with ideas of “standards-based instruction” and described programs they thought were 
needed to support these ideas, the ability to put into place the programs seen as needed was 
limited due to financial resources. One interesting finding in this area was that, while low 
income districts may have administrators that have very high human capacity (an-understanding 
of the reform ideas and content knowledge, as well as the will to pursue change), they are limited 
in what they can do because of low teacher will and capacity, lack of financial resources and 
number of district-level staff Still, places like Ridley are trying hard to overcome the problem 
of teacher will by forming partnerships with many outside experts and organizations and 



Teaching the Test to the Teachers 



24 



DRAFT 



corporations to provide the funding, materials, people, and ideas to help their teachers learn new 
ideas and strategies. This means, low-income districts may have to do change in a piecemeal 
way, where as higher income districts that have sufficient will and human capacity can do more 
systemic change or system wide change. 

Discussion 

The rhetoric behind using assessment to drive instructional change focuses on the nature of 
the state testing policy and accountability. However, this paper shows how six districts operating 
within the same set of state policies responded quite differently. Our findings are consistent with 
research on district reform (c.f Fairman & Firestone, 2001; Spillane & Thompson, 1997), in 
finding that districts have varying capacity for reform, but enhance that literature by focusing on 
strategies used in responding specifically to state standards and testing. Overall, we found that 
districts’ responses to standards and testing were heavily influenced by their capacity to support 
more general reform efforts. In the one district where responses were most consistent with the 
expectations of state reform advocates, a combination of factors were involved. The human 
capacity and will were very high in Sunset, along with some growing pressure from parents 
about test scores, and that all together meant they pursued change that was more systemic and 
deeper. However, the primary catalyst was teachers' and administrators' own beliefs and 
understandings of the reform ideas, which was lacking in a district like Newtown. 

If district administrators do not have a strong interest in reform and a reasonable 
understanding of the purpose of the assessment and the types of changes in teachers’ pedagogical 
content knowledge, then they are unlikely to respond to state expectations in substantial and 
sustained ways. As well, they must have the financial resources and the social networks and 
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tools necessary for such change. Administrators need to not only understand reform ideas but 
also believe that teachers need to engage in substantial learning in order to implement them. 
While some argue that “teaching to the test” (Resnick & Resnick, 1992) can be a beneficial 
strategy if the tests themselves are good, the kinds of district strategies consistent with teaching 
to the test found in this study varied considerably. In keeping with this theme, the will for 
reform needs to be linked in part to internal “buy-in” or support for state-supported changes, not 
just external pressures linked with test scores. 

Thus, in any income category, the variation in administrators' will and capacity (knowledge 
and understandings of the reform) and teachers' will and capacity are key to the problem of 
driving change through testing and standards. In a few cases, a few teachers have helped to move 
their district toward change by bringing back ideas from professional development experiences. 
But, in most cases, district leaders needed to communicate a clear goal for moving in this 
direction and use a combination of pressure and support to move the more reluctant (and usually 
older, veteran teachers) to make an effort to learn new content and change practice. In the 
absence of this leadership or understanding, change efforts are really just superficial, where they 
match up the topics and lingo of the standards to their curriculum documents, but don't really 
change what and how teachers are learning and the materials they have to use, or provide on- 
going support. 

If state policy makers wish to use assessment as a tool to promote instructional change and, 
ultimately, enhanced student learning, they need to recognize that how districts interpret and 
respond to new state policies is an important intervening factor in their efforts. While individual 
teachers may, through their own independent learning, develop the pedagogical content 
knowledge promoted by reformers, the support of districts in this area could be a significant asset 
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for policy makers. Building district capacity in all its forms may be a critical (although likely not 
sufficient) mechanism for using state assessments as an effective policy tool. 
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Tables 

Table 1: District Characteristics 



District 


DFG 


Size 


Grades 


Rural/Suburban/ 

Urban 


District 

Administrators 

Interviewed 


Ridley 


B 


5,000-10,000 


K-12 


Urban 


3 


Cedarville 


B 


8,000-12,000 


PreK-12 


Rural/Suburban 


5 


Hillview 


CD 


5,000-10,000 


PreK-12 


Urban 


4 


Sunset 


I 


1,000-2,000 


K-8 


Rural 


2 


Newtown 


I 


1,500-2,500 


K-8 


Suburban 


2 


Willis 

Creek 


I 


2,000-4,000 


K-8 


Suburban 


3 



Table 2: Types of District Responses to Standards and Testing 





The Process of Change 


Tools Used in Change 


District-Driven 




Combination of district and 






teacher leadership 


Primarily test- 


Newtown 






preparation/ “test-besting” 


Willis Creek 






Combination of test-prep 


Hillview 






and system-wide reform 


Ridley 




Cedarville 


Primarily focused on 
system-wide reform 




Sunset 
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Table 3: Tools used in District Responses to Testing 
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Table 5: Districts’ Approach to Reform and Will and Capacity 
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